Automatic Divide-and Conquer Using Population and Ensembles

Real-world problems are often too large and complex for a single monolithic system to solve. In practice, the divide-and-conquer strategy has often been used to decompose a large and complex problem into smaller tractable sub-problems and then solve them. However, good decomposition of large and complex problems requires experienced human experts and rich prior domain knowledge, which are usually unavailable for real-world problems. This talk explores some of our research efforts towards an adaptive approach to divide-and-conquer in the design of machine learning systems, e.g., evolutionary and neural learning systems. The basic idea is to move away from designing a single monolithic system that would solve a large and complex problem, and to employ a population of simpler sub-systems that will cooperatively solve the problem. In such populationsbased systems, each sub-system will be simpler and easier to learn than the monolithic system. The integrated system based on the whole population can generalise better than any single subsystems in the population. In particular, by evolving and training a team of specialists from random initial conditions, we were able to ”decompose” a large and complex problem into simpler ones and solve them without human intervention [1]. Two major approaches will be described. One uses the population structure in evolutionary algorithms, where individuals in a population are evolved into species (i.e., specialists for solving sub-problems) [2]. The other uses neural network ensembles in which individual neural networks learn to differentiate from and cooperate with each other [3, 4]. A constructive algorithm for designing ensembles as well as individual neural networks will be introduced [5].